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EXPLORATORY  ANALYSIS  OF  PERSONNEL  AND  PERFORMANCE: 

MOBILE  SUBSCRIBER  EQUIPMENT  FOLLOW-ON  OPERATIONAL  TEST  AND  EVALUATION 


Summary 

This  exploratory  research  used  data  from  the  MSE  Follow-On  Operational 
Test  and  Evaluation  to  (a)  describe  the  personnel  characteristics  of  MSE 
operators,  (b)  describe  operator  performance  on  critical  MSE  tasks,  and  (c) 
identify  personnel  variables  which  could  be  used  to  account  for  variability  in 
operator  performance  or  predict  operator  performance.  Ten  personnel 
variables  and  21  critical  tasks  were  considered.  Additional  data  included 
test  performance  data,  operator  errors  and  subjective  workload  ratings. 
Findings  can  be  applied  to  the  design  of  future  tests  and  to  design  of 
prototype  algorithms  for  personnel  selection  and  prediction  of  operator 
performance. 


System  Description 

The  Mobile  Subscriber  Equipment  (MSE)  is  a  new  battlefield  communications 
system  slated  to  become  the  backbone  of  U.S.  Army  corps  and  division 
communications.  The  MSE  system  is  a  Non-Developmental  Item  procured  through 
GTE  and  is  currently  being  fielded  at  Fort  Hood,  Texas  and  Fort  Gordon, 
Georgia.  It  is  anticipated  that  approximately  18,000  soldiers  will  be 
MSE-trained,  with  fielding  through  FY93  to  Involve  over  50  signal  battalions. 

MSE  Integrates  the  functions  of  transmission,  switching,  control, 
communications  security,  and  both  voice  and  data  terminal  equipment  into  one 
system.  As  a  switched  telecommunications  system,  MSE  is  extended  by  mobile 
radiotelephone  and  wire  access.  The  heart  of  the  system  is  node  center 
switches  (NCS).  These  centers  provide  connections  to  large  extension  switches 
(LEN),  small  extension  switches  (SEN),  and  radio  access  units  (RAU)  and  are 
linked  together  by  line-of-sight  (LOS)  radio  trunking.  Extension  switches 
allow  wire  line  terminal  subscribers  (telephone,  facsimile,  and  data)  to  enter 
the  system.  Radio  access  units  provide  mobile  radiotelephone  users  an 
interface  to  MSE  and  the  ability,  through  an  NCS,  to  communicate  with  other 
mobile  and  wire  telephone  users.  System  control  centers  (SCC)  provide 
processing  capability  for  data  inputs  to  aid  in  network  management.  MSE 
subscriber  service  is  facilitated  by  user-owned  equipment  which  Includes 
digital  nonsecure  voice  terminals  (DNVT)  and  mobile  subscriber  radiotelephone 
terminals  (MSRT) .  MSE  is  capable  of  interfacing  with  other  communications 
systems,  to  include  combat  net  radio  (CNR)  users,  NATO  and  allied  military 
systems,  and  host  nation  commercial  telephone  systems.  Major  components  of 
MSE  (NCS,  SCC,  LEN,  SEN,  RAU,  and  LOS)  constitute  self-contained  assemblages 
configured  in  wheeled  vans.  Major  auxiliary  equipment  for  MSE  includes 
generators,  15M  and  30M  masts,  and  various  antennas.  More  detailed 
information  on  the  MSE  system  is  available  in  FM  11-999E,  "Mobile  Subscriber 
Equipment  (MSE)  Architecture." 
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Background 


During  the  summer  and  fall  of  1988,  the  MSE  Follow-On  Test  and  Evaluation 
(FOTE)  was  conducted  at  Fort  Hood,  Texas.  This  was  the  first  U.S.  Army  test 
of  MSE  as  a  non-developmental  item.  This  test  was  in  conjunction  with  the 
initial  fielding  of  MSE  with  the  13th  Signal  Battalion,  1st  Cavalry  Division. 
The  test  was  conducted  by  Electronic  Proving  Ground  (EPG) ,  and  the  U.S.  Army 
Operational  Test  and  Evaluation  Agency  (OTEA)  was  the  Independent  evaluator. 

A  majority  of  manpower,  personnel,  training,  human  factors  engineering,  system 
safety,  and  health  hazards  (MANPRINT)  data  were  collected  and  processed  by  a 
contractor.  A  comprehensive  report  of  FOTE  findings,  to  include  MANPRINT,  is 
available  from  OTEA. 

In  June  1988,  the  Fort  Gordon  Field  Unit  (FGFU)  of  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  requested  support  from 
the  ARI  Fort  Hood  Field  Unit  (FHFU)  on  the  completion  of  work  directed  by 
General  Thurman  in  November  1987.  The  requested  support  Included  analysis  of 
MSE  training  data,  analysis  of  MSE  FOTE  data  reflecting  on  personnel  and 
training  considerations,  and  development  of  a  performance  test  for  the 
standardized  evaluation  of  MSE  soldiers.  This  research  project's  plan  was 
revised  in  March  1989  to  place  increased  emphasis  on  MSE  operator  (31D  and  31F 
MOSs)  performance  relative  to  selection  considerations  and  development  of  both 
training  and  SQT  standards.  Additionally,  in  March  1989  EPG  requested  ARI 
assistance  in  developing  operator  performance  algorithms  which  could  be 
integrated  into  an  existing  computer  simulation  of  MSE  hardware  model.  In 
April  1989,  OTEA  concurred  with  the  proposed  ARI  research  effort  as  viable  and 
of  interest  to  them  and  provided  all  requested  MANPRINT  MSE  FOTE  data. 

The  purpose  of  this  Interim  Report  is  to  consolidate  relevant  MSE  FOTE 
operator  personnel  and  performance  data,  describe  parameters  of  these  data, 
and  provide  data  analyses  which  could  Impact  personnel  selection  and 
performance  standards.  It  is  suggested  that  this  report  and  the  data 
presented  could  be  used  to:  (a)  develop  prototype  personnel  selection 
criteria  for  future  validation  testing;  (b)  develop  prototype  Initial 
performance  distributions  and  standards;  (c)  develop  prototype  operator 
performance  algorithms  which,  when  validated,  could  be  integrated  into  the  EPG 
MSE  model;  and  (d)  suggest  areas  of  potential  interest  for  future  (Follow-On 
Evaluation,  Jan-Mar  90)  MSE  testing.  As  the  collection  of  the  data  presented, 
described,  and  analyzed  in  this  report  was  not  conducted  or  supervised  by  ARI, 
no  statements  of  data  integrity  can  be  made  or  are  implied  by  this  report. 

Method 


Participants 


Personnel  and  performance  data  were  provided  through  OTEA  for  280  soldiers 
transitioned  into  31D,  31F,  or  31W  MOSs  during  the  conduct  of  the  MSE  FOTE. 

For  this  group  of  soldiers,  86%  were  men,  159  (57%)  were  31Ds,  88  (31%)  were 
31Fs,  and  33  (12%)  were  31Ws.  Feeder  MOSs  Included  25B,  31C,  31G,  31K,  31L, 
31M,  31N,  31Q,  31V,  31Y,  31Z,  36C,  36L,  36M,  and  72E;  the  largest  proportion 
of  soldiers  (56%)  came  from  the  31M  MOS.  Ages  ranged  from  19  to  45  years, 
time  in  service  ranged  from  1  to  20,  and  years  of  education  ranged  from  9  to 
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17.  Standardized  test  score  ranges  within  this  group  were  from  16  to  99  on 
the  AFQT,  59  to  155  on  the  ASVAB  GT,  and  76  to  141  on  the  ASVAB  EL. 

Composition  of  Data 


A  wide  range  of  both  personnel  and  performance  data  on  these  MSE-trained 
soldiers  was  obtained.  For  each  soldier,  the  following  personnel  data  were 
requested:  pre-transition  MOS,  MSE  MOS,  DOB,  years  of  education,  handedness, 
gender,  date  of  PMOS,  years  of  service,  SQT  score,  ASVAB  GT,  ASVAB  EL,  and 
AFQT.  Not  all  data  were  available  for  each  soldier,  and  some  data  were 
suspected  to  be  in  error.  For  each  soldier,  the  following  performance  data 
were  requested:  pretraining  test  score;  posttraining  test  score;  and  task 
completion  times  for  troubleshooting,  switch  initialization,  antenna 
orientation,  essential  user  bypass  (EUB),  SCC  VDU/GDU  operation,  SCC 
initialization,  SCC  digital  map  loading,  SCC  frequency  plan  management, 

NCS/LEN  set  up,  LOS  set  up,  RAU  set  up,  SEN  set  up,  SCC  set  up,  MSRT 
installation,  key  loading  RT-1539  (MSRT/RAU),  key  loading  KY-68  (MSRT/RAU) , 
key  loading  KG-94  (LEN/NCS/SEN/RAU) ,  key  loading  KG-82  (NCS/LEN),  key  loading 
KY-57  (NCS/LEN/ SEN/LOS/RAU),  and  key  loading  KY-90  (LEN/SEN).  In  addition, 
data  were  requested  on  operator  errors  and  subjective  workload  ratings  by 
task.  Not  all  data  were  available  for  each  soldier. 

No  single  soldier  or  single  team  of  soldiers  would  be  expected  to  perform 
all  listed  tasks,  as  tasks  are  MOS-speciflc  (31D,  31F,  and  31W) .  The  small 
amount  of  data  available  on  some  tasks  precluded  meaningful  statistical 
considerations  of  those  tasks,  and  regression  analyses  applied  to  personnel 
and  performance  data  crosswalks  were  appreciably  restricted  (reduced  ^s)  by 
missing  data.  While  operator  error  data  in  terms  of  error  nature  were 
obtained,  no  frequency  or  rate  data  were  available.  The  workload  ratings 
provided  by  task  were  small  in  number  and,  for  present  purposes,  only  the 
ratings  from  the  Physical  Effort,  Mental  Effort,  and  Overall  Workload  scales 
were  considered  in  presentations  and  analyses. 

Procedures 


All  data  were  obtained  from  EPG  through  OTEA  and  were  provided  as 
printouts.  Prior  to  reconfiguration  and  entry  into  an  ARI  data  base,  some 
(approximately  3%)  of  the  personnel  data  were  deleted  due  to  Integrity 
challenges  (e.g.  impossible  scores  on  standardized  tests,  improbable  DOBs 
[ages]  for  soldiers,  or  nonexistent  PMOSs).  A  data  screen  to  accommodate 
entry  of  all  possible  data  for  each  soldier  was  constructed  and  used  to  enter 
accepted  data.  The  total  possible  number  of  data  points  for  each  soldier  was 
80  assuming  each  soldier  had  all  personnel  measures,  performed  all  tasks,  and 
completed  all  workload  scales  for  each  task. 

Data  were  processed  by  a  IBM  4381  mainframe  computer  using  the 
Statistical  Analysis  System  (SAS)  Version  5  software.  The  predominant 
programs  used  provided  descriptive  statistics  on  distributions  (PROC  MEANS  and 
PROG  FREQ),  PMOS-comparatlve  statistics  on  task  performances  (PROC  GLM) , 
relationships  between  variables  or  measures  (PROC  CORR) ,  and  regression 
techniques  which  Identify  combinations  of  variables  which  could  aid  in 
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accounting  for  variability  in  (PROC  RSQUARE)  or  predicting  (PROC  STEPWISE) 
task  performance. 

All  descriptive,  correlational,  and  inferential  data  analysis  outcomes 
are  presented  in  tabular  fashion.  The  provision  of  means  and  standard 
deviations  for  all  appropriate  personnel  data  distributions  and  for  critical 
task  performance  distributions  allows  for  easy  identification  of  any 
percentile  **cuts"  desired:  the  5th  percentile  equates  to  a  ^  score  of  -1.65 
and  the  95th  percentile  equals  a  ^  score  of  +1.65  assuming  that  the  underlying 
measures  are  essentially  normally  distributed.  Most  personnel  variables  do 
not  grossly  violate  this  assumption,  though  small  Ns  for  many  of  the  critical 
task  performances  preclude  the  possibility  of  a  normal  distribution. 

Caution  must  be  exercised  Interpreting  operator  task  performance  data  as 
presented.  Many  operator  tasks  were  team  efforts.  Further,  across  any  given 
task,  teams  were  unequal  in  size  and  the  composition  (membership)  of  a  team 
could  change.  The  obtained  data  accorded  each  team  member  was  that  team^s 
total  time  to  completion,  and  no  fair  comparison  of  soldiers^  performance 
could  be  made  using  the  original  data.  To  produce  some  standardization  which 
would  allow  more  meaningful  representations  or  comparisons  of  a  soldier^s  task 
performance,  these  data  were  modified:  each  soldier  was  awarded  the  total 
man-minutes  (total  time  multiplied  by  number  of  team  members)  needed  by  the 
team  to  complete  a  task.  If  a  soldier  performed  the  same  task  more  than  once, 
the  mean  of  all  performances,  in  man-minutes,  was  used  as  the  best  measure  of 
that  soldier's  performance.  This  single  value  was  entered  in  the  performance 
data  base  to  represent  that  soldier.  Hence,  all  performance  data  for  critical 
tasks  used  and  reported  in  this  research  effort  represent  the  total  number  of 
man-minutes  needed  by  a  team  (1  to  5  soldiers)  to  complete  the  task. 

Findings 

This  research  effort  is  exploratory.  It  was  designed  primarily  to 
provide  insight  into  possible  concerns  or  interests  for  future  tests  and 
evaluations  and  yield  prototype  algorithms  for  operator  selection  and  task 
performance  criteria.  For  a  number  of  critical  tasks,  performance  data  were 
too  few  to  justify  any  statistical  processing,  and  for  some  tasks,  the 
variability  of  performance  was  so  great  as  to  preclude  accountability  by  any 
variable  in  the  data  base.  Also,  it  was  noted  that  performance  on  one 
critical  task  was  often  totally  unrelated  (or  negatively  related)  to 
performance  on  another  critical  task  by  the  same  group  of  operators.  In  most 
cases  involving  regression  analyses  of  personnel  variables  on  task 
performance,  sample  sizes  represented  in  outcomes  were  small  because  any 
missing  value  resulted  in  the  entire  observation  being  discarded. 

Tables  2-7  provide  descriptive  information  on  the  personnel 
characteristics  of  operators,  as  summarized  in  Table  1.  Table  8  provides 
descriptive  information  on  operator  performance  on  critical  tasks,  and  Table  9 
offers  a  comparison  of  MSE-feeder  MOSs  in  terms  of  critical  task  performances. 
Tables  10  and  11  show  significant  relationships  between  and  among  both 
personnel  variables  and  critical  task  performances  and  the  derivation  of 
personnel  variable  predictors  of  performance.  Table  12  compares 
communications  procedures  pre-  and  post test  performances  and  MSE  MOS  groups  on 
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Table  1  -  MSE  PERSONNEL  PROFILE 


Personnel  Variable 

Number 

Mean 

Standard 

Range 

Deviation 

MOS  =  31D 


Years  of  Age 

159 

25.7 

5.2 

19  -  45 

Years  of  Education 

148 

12.2 

.7 

11  -  16 

Years  of  Service 

159 

4.5 

3.8 

1  -  18 

AFQT  Score 

152 

51.0 

18.5 

16  -  99 

ASVAB  GT  Score 

147 

104.2 

11.7 

59  -  155 

ASVAB  EL  Score 

142 

102.6 

10.1 

80  -  132 

Gender  (Men) 

89% 

N/A 

N/A 

N/A 

Handedness  (Right) 

91% 

N/A 

N/A 

N/A 

MOS  =  31F 


Years  of  Age 

88 

25.9 

4.5 

19  -  40 

Years  of  Education 

83 

12.2 

.9 

9-16 

Years  of  Service 

88 

4. 1 

3.2 

1  -  13 

AFQT  Score 

83 

54.7 

16.4 

25  -  98 

ASVAB  GT  Score 

84 

105.5 

11.5 

63  -  143 

ASVAB  EL  Score 

82 

104.5 

10.8 

76  -  141 

Gender  (Men) 

76% 

N/A 

N/A 

N/A 

Handedness  (Right) 

90% 

N/A 

N/A 

N/A 

MOS  =  31W 


Years  of  Age 

33 

36.2 

4.4 

24  -  45 

Years  of  Education 

29 

13.0 

1.4 

12  -  17 

Years  of  Service 

33 

15.0 

4.3 

2-20 

AFQT  Score 

27 

56.9 

22.3 

18  -  95 

ASVAB  GT  Score 

28 

112.9 

13.2 

92  -  143 

ASVAB  EL  Score 

14 

111.2 

18.3 

76  -  138 

Gender  (Men) 

100% 

N/A 

N/A 

N/A 

Handedness  (Right) 

88% 

N/A 

N/A 

N/A 

Composite  Group 


Years  of  Age 

280 

27.0 

5.9 

19  -  45 

Years  of  Education 

260 

12.3 

.9 

9-17 

Years  of  Service 

280 

5.6 

5.0 

1  -  20 

AFQT  Score 

262 

52.8 

18.4 

16  -  99 

ASVAB  GT  Score 

259 

105.6 

12.0 

59  -  155 

ASVAB  EL  Score 

238 

103.8 

11.1 

76  -  141 

Gender  (Men) 

86% 

N/A 

N/A 

N/A 

Handedness  (Right) 

90% 

N/A 

N/A 

N/A 
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Table  2  -  DISTRIBUTION 

OF  GENDER  AND  LATERALITY  AMONG  MSE  MOSs 

MOS 

Gender 

Laterality 

Men 

Women 

Right 

Left 

31D 

89% 

11% 

91% 

9% 

31F 

76% 

24% 

90% 

10% 

31W 

100% 

0% 

88% 

12% 

Composite 

86% 

14% 

90% 

10% 
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Table  3  -  DISTRIBUTION  OF  STANDARDIZED  TEST  SCORES  AMONG  MSE  PERSONNEL 


Armed 

Forces  Qualification 

Test 

(AFQT) 

MOS 

10-19 

20 

1-29 

30-39  ^ 

40-49 

50-59 

60-69 

70-79 

80-89 

90+ 

31D 

2% 

8% 

22% 

22% 

13% 

13% 

12% 

5% 

3% 

31F 

0% 

4% 

16% 

19% 

24% 

20% 

6% 

10% 

1% 

31W 

11% 

0% 

7% 

19% 

15% 

15% 

15% 

11% 

7% 

Total 

2% 

6% 

19% 

21% 

16% 

16% 

10% 

7% 

3% 

ASVAB 

General 

Technical  (GT) 

MOS 

50-59 

60 

1-69 

70-7S 

1  80-89  90-99 

100-109 

110-119  120-129 

130-139  140+ 

31D 

n 

0% 

0% 

9% 

24% 

35% 

23% 

7% 

0% 

1% 

31F 

0% 

^% 

0% 

5% 

26% 

31% 

27% 

8% 

0% 

1% 

3IW 

0% 

0% 

0% 

0% 

14% 

32% 

21% 

21% 

7% 

4% 

Total 

<n 

<1% 

0% 

7% 

24% 

34% 

24% 

9% 

1% 

1% 

ASVAB  Electronics  (EL) 

MOS 

70 

-79 

1  80-89 

90-99 

100-109  110-119  120-129  130-139 

140+ 

3ID 

0% 

6% 

39% 

28% 

22% 

4% 

1% 

0% 

3IF 

^% 

4% 

30% 

38% 

18% 

7% 

0% 

1% 

3IW 

7% 

7% 

7% 

21% 

21% 

21% 

14% 

0% 

Total 

n 

5% 

34% 

31% 

21% 

6% 

2% 

<1% 
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Table  4  -  DISTRIBUTION  OF  EDUCATION  AMONG  MSE  PERSONNEL 


MOS 

9 

10 

11 

Years  of  Education 

12  13  14 

15 

16 

17 

31D 

0% 

0% 

n 

86? 

5? 

4? 

1? 

1? 

0? 

31F 

^% 

0% 

2% 

84? 

5? 

4? 

0? 

4? 

0? 

31W 

0% 

0% 

0% 

52? 

21? 

21? 

0? 

0? 

7? 

Composite 

<^% 

0% 

2% 

82? 

7? 

6? 

1? 

2? 

1? 
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Table  5  -  DISTRIBUTION  OF  TIME  IN  SERVICE  AMONG  MSE  PERSONNEL 


MOS 

1  -  3 

4-6 

Years 

7-9 

of  Service 
10  -  12 

13  -  15 

16  -  18 

19  -  21 

31D 

m 

27% 

12% 

8% 

3% 

1% 

0% 

31F 

50% 

26% 

17% 

6% 

1% 

0% 

0% 

31W 

6% 

0% 

0% 

12% 

33% 

30% 

18% 

Composite 

44% 

24% 

12% 

8% 

6% 

4% 

2% 
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Table  6  - 

DISTRIBUTION 

OF  AGE 

AMONG  MSE 

MOSs 

MOS 

17-19 

20-24 

Age 

25-29 

Group 

30-34 

35-39 

40-44 

45-49 

31D 

5% 

45% 

30% 

12% 

6% 

2% 

1% 

31F 

n 

43% 

39% 

13% 

3% 

1% 

0% 

31W 

0% 

3% 

3% 

27% 

45% 

18% 

3% 

Composite 

3% 

39% 

30% 

14% 

10% 

4% 

1% 

10 


Table  7  -  DISTRIBUTION  OF  FEEDER  MOSs  AMONG  MSE  MOSs 


MSE  MOS 

Feed  MOS 

31D 

(N) 

31F 

(N) 

31W 

(N) 

COMPOSITE 

(N) 

25B 

0% 

0 

0% 

0 

6% 

2 

1% 

2 

31C 

9% 

15 

8% 

7 

3% 

1 

8% 

23 

31D* 

1% 

2 

0% 

0 

0% 

0 

1% 

2 

31F* 

0% 

0 

2% 

2 

0% 

0 

1% 

2 

31G 

0% 

0 

0% 

0 

3% 

1 

<1% 

1 

31K 

9% 

15 

0% 

0 

0% 

0 

5% 

15 

31L 

5% 

8 

5% 

4 

3% 

1 

5% 

13 

31M 

56% 

89 

55% 

48 

0% 

0 

49% 

137 

3  IN 

5% 

8 

^5% 

13 

3% 

1 

8% 

22 

31Q 

^% 

1 

0% 

0 

0% 

0 

<1% 

1 

31V 

^% 

1 

0% 

0 

0% 

0 

<1% 

1 

31W* 

0% 

0 

0% 

0 

3% 

1 

<1% 

1 

31Y 

0% 

0 

0% 

0 

58% 

19 

7% 

19 

31Z 

0% 

0 

0% 

0 

15% 

5 

2% 

5 

36C 

0% 

0 

n 

1 

0% 

0 

<1% 

1 

36L 

1% 

1 

0% 

0 

0% 

0 

<1% 

1 

36M 

3% 

5 

w% 

9 

0% 

0 

5% 

14 

72E 

9% 

14 

5% 

4 

6% 

2 

7% 

20 

TOTAL 

159 

88 

33 

280 

*  Presumed  incorrect  prior  MOS  entry  on  personnel  data  form 
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Table  8  -  SUMMARY  OF  MSE  OPERATOR  CRITICAL  TASK  PERFORMANCES* 


Nature  of  Task 

N 

Primary 

MOS 

Mean  Time 
(man/min) 

Standard 

Deviation 

Range 

Troubleshooting 

18 

31D 

126.0 

127.9 

16-470 

Switch  Initialization 

10 

31F 

137.7 

141.7 

16-372 

Antenna  Orientation 

92 

31D 

56.5 

94.2 

1-465 

Essential  User  Bypass 

9 

31F 

14.6 

5.7 

6-24 

see  VDU/GDU  Operation 

3 

31F 

27.0 

N/A 

8-60 

see  Initialization 

5 

31W 

13.0 

N/A 

10-18 

NCS/LEN  VDU  Operation 

18 

31F 

61.0 

47.2 

8-177 

NCS/LEN  Set  Up 

28 

31F/W 

2405.5 

2265.5 

493-5776 

LOS  Set  Up 

87 

31D 

150.7 

79.9 

30-408 

RAU  Set  Up 

24 

31D 

181.9 

60.3 

48-264 

SEN  Set  Up 

42 

31F 

245.9 

165.5 

35-634 

sec  Set  Up 

7 

31W 

217.9 

155-330 

Key  Load  (KY-68) 

2 

31D 

1.0 

N/A 

0 

Key  Load  (KG-82) 

2 

31F 

6.5 

N/A 

5-8 

Key  Load  (KG-94) 

22 

31F 

2.4 

2.7 

1-11 

Key  Load  (KY-57) 

19 

31D/F 

1.1 

.2 

1-2 

Key  Load  (KY-90) 

3 

31F 

57.0 

N/A 

7-108 

Most  of  the  listed  tasks  constituted  team  efforts  and  some  tasks  were 
conducted  several  times  by  a  team,  though  teams  often  changed  personnel  and 
size.  As  individual  soldier  data  were  desired,  the  total  time  for  the  team  to 
complete  the  task  was  multiplied  by  the  number  of  team  members,  yielding  the 
total  man-minutes  required  to  complete  the  task.  This  "total  team  man- 
minutes"  time  for  completing  a  given  task  was  assigned  to  each  team  member 
and  averaged  across  repeated  measurements  for  that  team  member  to  compensate 
for  team  composition  differences.  This  resulted  in  a  task  performance  time 
for  each  operator  reflecting  the  mean  man-minutes  for  team  task  completion 
with  which  a  given  soldier  was  associated. 
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Table  9  -  SUMMARY  ANALYSIS  OF  PRIOR  MOS  EFFECT  ON  MSE  TASK  PERFORMANCE 


Task 

N 

Prior  MOSs 
Involved 

F 

L 

Significant 

Difference* 

Troubleshooting 

18 

31K,  31L,  31M, 
36M,  72E 

3.63 

.0337 

31M  < 
31M  < 

31L, 

31K 

Switch  Initialization 

10 

31C,  31M,  31N. 
31Y 

9.03 

.0121 

31C  < 
31C  < 
31M  < 
31M  < 

31N, 

31Y, 

31Y, 

31N 

Antenna  Orientation 

92 

31C,  31D,  31K, 
31L,  31M,  31N 
31V,  36L,  36M, 
72E 

0.46 

.8972 

None 

Perform  EUB 

9 

31C,  31M,  31N 

1.70 

.2601 

None 

NCS/LEN  VDU  Operation 

18 

31C,  31M,  31N 
31Y,  72E 

1.06 

.4139 

None 

NCS/LEN  Set  Up 

28 

31C,  31M,  31N 
31Y,  36M,  72E 

2.09 

.1054 

31M  < 
72E  < 

31N 

31N 

LOS  Set  Up 

87 

31C,  31D,  31K, 
31L,  31M,  31N, 
31Q,  31V,  31Y, 
36M,  72E 

1.13 

.3492 

31C  < 
31L  < 
31M  < 
31N  < 
31V  < 
36M  < 

31Y 

31Y 

31Y 

31Y 

31Y 

31Y 

RAU  Set  Up 

24 

31C,  31K,  31M, 
31N,  31V,  36M 

1.01 

.4385 

None 

SEN  Set  Up 

42 

31C,  31F,  31L, 
31M,  31N,  36M 
72E 

1.79 

.1297 

31M  < 
36M  < 

31F»» 

31F«* 

see  Set  Up 

7 

25B,  31C,  31L, 
31N,  31Y 

1.10 

.5277 

None 

Key  Load  (KG-94) 

22 

31C,  31K,  31L, 
31M,  31N,  36M, 
72E 

1.02 

.4494 

31C  < 

72E 

Key  Load  (KY-57) 

19 

31C,  31D,  31K, 
31M,  31Y,  72E 

0.07 

.9960 

None 

•Based  on  exploratory  analyses  by  ^  tests  (£<.05).  Later  more  definitive 
analyses  will  utilize  the  more  stringent  Scheffe's  Test. 

♦*31F  (and  31D)  pre- transition  MOSs  likely  are  incorrect  data  entries. 
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Table  10  -  SIGNIFICANT  RELATIONSHIPS  OF  PERSONNEL  AND  PERFORMANCE  VARIABLES* 


Correlations  Among  Personnel  Variables  (8) 

Variables 

r^ 

N 

L 

ASVAB  GT  +  AFQT 

.680 

243 

.0001 

Years  Education  +  Years  in  Service 

.104 

260 

.0934 

ASVAB  GT  +  ASVAB  EL 

.547 

231 

.0001 

ASVAB  EL  +  Gender 

-.138 

238 

.0329 

ASVAB  GT  +  Years  Education 

.256 

240 

.0001 

Years  in  Service  +  Age 

.853 

280 

.0001 

Age  +  ASVAB  GT 

.151 

259 

.0153 

Years  Education  +  Age 

.236 

260 

.0001 

AFQT  +  ASVAB  EL 

.623 

225 

.0001 

ASVAB  EL  +  Years  Education 

.191 

220 

.0046 

AFQT  +  Years  Education 

.189 

243 

.0031 

Correlations  Among  Critical 

Task  Performances  (12) 

Variables 

r^ 

N 

L 

Troubleshooting  +  Switch  Initialization 

.998 

3 

.0383 

LOS  Set  Up  +  Antenna  Orientation 

.213 

63 

.0940 

NCS/LEN  Set  Up  +  Switch  Initialization 

.729 

8 

.0402 

RAU  Set  Up  +  Antenna  Orientation 

-.975 

6 

.0009 

LOS  Set  Up  +  Key  Load  (KG-94) 

-.999 

3 

.0334 

Correlations  Between  Personnel 

Variables  and 

Performances 

Variables 

r 

N 

L 

Age  +  Switch  Initialization 

.554 

10 

.0965 

AFQT  +  Troubleshooting 

.476 

18 

.0457 

Gender  +  Key  Load  (KG-94) 

.385 

22 

.0768 

ASVAB  GT  +  LOS  Set  Up 

.271 

80 

.0150 

Years  Education  +  NCS/LEN  Set  Up 

.378 

26 

.0570 

Age  +  RAU  Set  Up 

-.371 

24 

.0742 

AFQT  +  LOS  Set  Up 

.295 

82 

.0072 

ASVAB  EL  +  LOS  Set  Up 

.208 

80 

.0637 

Years  of  Service  +  Key  Load  (KG-94) 

.653 

22 

.0010 

Handedness  +  Switch  Initialization 

.581 

10 

.0781 

Age  +  Key  Load  (KG-94) 

.628 

22 

.0017 

*  Personnel  correlations  involved  28  possible  unique  combinations, 
critical  task  performance  correlations  involved  34  possible  unique 
combinations,  and  correlations  between  personnel  variables  and  performances 
involved  96  possible  unique  combinations.  Some  possible  combinations  were 
precluded  due  to  insufficient  data  (N<3).  Non-significant  correlations 
(p>.05)  are  not  listed. 
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Table  11  -  OPERATOR  PERFORMANCE  PREDICTOR  PERSONNEL  VARIABLES* 


Task 

Best  Combination  of  Predictors 

N 

Rj-Square** 

Troubleshooting 

Age  +  Education  +  GT  +  EL 

16 

.401 

Switch  Initialization 

Age  +  AFQT  +  GT  +  EL  +  Gender  + 

Hand 

9 

.999 

Antenna  Orientation 

Age  +  GT  +  EL  +  Hand 

75 

.056 

Perform  EUB 

Age  +  Years  Service  +  AFQT  +  GT 

8 

.970 

NCS/LEN  VDU  Operation 

Education  +  AFQT  +  GT  +  EL 

13 

.267 

NCS/LEN  Set  Up 

Age  +  Education  +  AFQT  +  GT  +  Hand 

18 

.517 

LOS  Set  Up 

Years  Service  +  Education  +  AFQT  + 
EL 

68 

.152 

RAU  Set  Up 

Education  +  AFQT  +  GT  +  EL  +  Gender 

16 

.436 

SEN  Set  Up 

Years  Service  +  Education  +  AFQT  + 
EL  +  Hand 

35 

.225 

Key  Load  (KG-94) 

Years  Service  +  GT  +  EL  +  Hand 

20 

.606 

Key  Load  (KY-57) 

Years  Service  +  Education  +  AFQT  + 
GT  +  EL 

14 

.532 

*  Total  of  10  personnel  variables  were  available;  prior  MOS  was  not  used  as 
it  was  treated  elsewhere  and  while  gender  and  handedness  were  used ,  some 
statistical  liberties  were  applied  to  their  coding  and  processing.  For 
reasons  of  diminishing  returns,  a  maximum  of  5  variables  was  allowed  for  any 
listed  combination. 

**  R^square  may  be  interpreted  as  the  proportion  of  variance  in  a  task 
performance  which  can  be  accounted  for  by  knowledge  of  indicated  variables. 
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Table  12  -  COMPARISON  OF  COMMUNICATIONS  PROCEDURES  TEST  PERFORMANCE* 


Composite  Group 

Pretest 

Mean 

N_  Standard 
Deviation 

Posttest  Mean 

N  Standard  ^  £ 

Deviation 

43.48? 

254  14.79 

27  30.68  <.001 

MSE  MOS 

Comparisons  on 

Pretest 

F 

L 

Groups  Compared 

Difference 

Outcome 

48.86 

<.001 

31D,  31F 

1.05 

>.05 

No  Difference 

31D,  31W 

26.10 

<.05 

31W  >  31D 

31F,  31W 

25.05 

<.05 

31W  >  31F 

MSE  MOS  Comparisons  on 

Posttest 

F 

E. 

Groups  Compared 

Difference 

2»* 

Outcome 

18.33 

<.001 

31D,  31F 

5.98 

<.05 

31F  >  31D 

31D,  31W 

10.96 

<.05 

31W  >  31D 

31F,  31W 

4.98 

>.05 

No  Difference 

*  Test  scores  were  in  terms  of  percent  correct 
•*  Based  on  application  of  Scheffe's  Test 
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Table  13  -  BEST  AVAILABLE  PERFORMANCE  PREDICTION  ALGORITHM* 


Task 

N 

Prediction  Equation  ^-Square 

(Variables  and  Weights) 

F 

L 

Troubleshooting 

15 

10.64  GT  +  271.26  Education 
-  4245.68 

.370 

3.53 

.062 

Switch  Initialization 

10 

12.85  Age  +  193  Hand 
-  438.51 

.465 

3.05 

.112 

Antenna  Orientation 

68 

None 

Perform  EUB 

9 

-.20  AFQT  +  -.12  Pre  Test 
+  31.08 

.383 

1.87 

.234 

NCS/LEN  VDU  Operation 

13 

1.12  Pre  Test  +  5.58 

.197 

2.69 

.129 

NCS/LEN  Set  Up 

20 

533.07  Education  +  39.59  AFQT 
+  5078  Hand  -  93.21  Post  Test 
-  4724.88 

.584 

5.26 

.008 

LOS  Set  Up 

62 

.98  AFQT  +2.82  Pre  Test 
-13.29 

.240 

9.33 

<.001 

RAU  Set  Up 

14 

None 

SEN  Set  Up 

27 

114.15  Education  +  135  Hand 
-  5.92  Post  Test  -  865.98 

.252 

2.59 

.078 

Key  Load  KG-94 

18 

.50  Time  in  Service  +  .21 

.439 

12.53 

.003 

Key  Load  KY-57 

11 

-.02  EL  +  2.60 

.159 

1.71 

.224 

*  Variables  were  accepted  into  the  regression  model  only  if  they  met  the 
criterion  £_ of  F_<,50.  They  were  retained  in  the  prediction  equation  only  if 
they  met  the  criterion  of  FL.25  as  variables  were  added  to  the  regression 
model.  An  actual  prediction  equation  was  provided  only  if  the  final  ^-Square 
(proportion  of  performance  variance  accounted  for)  F  value  had  a  £<.2^.  For 
computational  purposes,  the  variable  on  hand  was  coded  right  =  1  and  left  =  2, 
units  of  education,  age,  and  time  in  service  were  years,  pre  and  posttest 
scores  were  in  percents,  and  AFQT  and  ASVAB  GT  were  in  original  standard 
scores. 
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Table  14  -  SUBJECTIVE  WORKLOAD  RATINGS  FOR  MSE  TASKS* 


Task  (Nominal) 

N 

Mean 

Mental  Effort 

Workload  Rating** 
Physical  Effort 

Overall*** 

30M  Mast  Erection 

6 

3.5 

3.0 

3.3 

RAU  Set  Up/Initialization 

9 

2.0 

3.2 

3.0 

Antenna  Orientation 

5 

2.6 

2.0 

3.0 

LOS  Set  Up/Initialization 

15 

2.1 

2.8 

2.4 

VDU  and  Switch  Operation 

5 

2.2 

2.0 

3.0 

Perform  EUB 

4 

2.3 

1.0 

1.8 

Establish  Links 

3 

2.0 

2.3 

2.7 

SEN  Set  Up/Initialization 

10 

1.9 

2.6 

2.2 

15M  Antenna  Set  Up 

3 

1.7 

1.7 

2.7 

Antenna  (unknown)  Set  Up 

6 

2.8 

3.8 

4.3 

Shelter  (unknown)  Set  Up 

6 

2.5 

3.7 

3.0 

Task  Performance  (Actual) 

Correlation 

with 

Workload  Ratings** 

N 

Mental  Effort 

Physical  Effort 

Overall 

Antenna  Orientation 

5 

-.912  (p<.( 

35) 

.410 

-.076 

NCS/LEN  VDU  Operation 

5 

.451 

-.215 

-.207 

LOS  Set  Up 

13 

-.472 

-.147 

o 

• 

1 

RAU  Set  Up 

9 

-.473 

-.358 

-.351 

SEN  Set  Up 

10 

1 

1 

1 

1  O 

1  VO 

1  OO 

1 

1 

1 

-.137 

-.  030 

*  Tasks  for  which  there  were  less  than  3  respondents  were  omitted . 

**  The  rating  scales  extended  from  "I”  (factor  barely  contributed  to  task's 
overall  workload)  to  "5”  (factor  is  the  primary  cause  contributing  to  overall 
workload  and  one  was  not  able  to  keep  up  with  work  required) .  Data  may  not  be 
valid  due  to  poor  definition  of  scale  value  ”1”  compared  to  other  scale 
values  and  to  small  samples. 

***  Refers  to  soldiers*  ratings  of  "Overall  Workload”  for  each  task  —  not 
to  average  of  Mental  Effort  and  Physical  Effort  ratings. 
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Table  15  -  MSE  OPERATOR  ERRORS* 


Nature  of  Error 


MOS  Involved 


DSVT  affiliation  31D 

Cables  (J4,J5-J8,  J9)  in  wrong  place  31Df  31F 

Wrong  entry  for  trans  frequency  3 IF,  31W 

Turned  switchboard  off  while  troubleshooting  31F 

Failed  to  check  oil  in  generator  31D 

Dropped  wrong  link  3 IF 

AC/DC  power  selector  switch  in  wrong  position  31D 

Wrong  variables  from  NCS  to  SEN  31F 

Did  not  turn  over  batteries  in  AKDC  3 IF 

Interconnecting  cable  not  properly  connected  31F 

Interconnection  cables  J4  and  J5  transposed  on 

NC  switch  31F 

Cable  hooked  up  at  wrong  hauk  31D 

Improper  receiver  frequency  on  GRC-224  31 D 

Accidental  change  of  frequency  on  UHF  radio  3 ID 

Grounding  to  generator  31 D 

Power  turned  off  and  system  shut  down  3 ID 

System  control  gave  wrong  frequencies  twice  31 D 

Switching  of  power  supply  3 ID 

NCS  ordered  shut  off  of  link  without  authority  31D 

Generator  ran  out  of  fuel  causing  loss  of  AC  power  3 ID,  3 IF 

NC  cut  off  system  without  notification  31F 

No  variables  to  load  into  equipment  3 IF 

Wrong  variable  T-key  31F 

Bad  profile  on  shot  31D 

Forgot  to  connect  power  cable  to  shelter  31 D 

Power  hum  on  OCU—rerun  coax  to  NC  3 ID 

Zeroized  TED  accidentally  31F 

Guy  winder  had  to  be  rewound/guy  wires  rerouted  3 ID 

Timing  card  switch  set  in  wrong  position  31D 

Generator  died  during  switch  in  shifts  31D 

SHF  antenna  fell— not  all  guy  ropes  attached  31F 


*  Frequency  of  error  data  not  available 
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those  performances*  Table  13  provides  the  best  available  prediction  equations 
for  performance  on  specific  critical  tasks  based  on  personnel  variables. 

Table  14  offers  descriptive  information  on  subjective  workload  ratings  for 
specific  critical  tasks  and  their  relationship  to  actual  task  performances* 
Table  15  lists  operator  errors  recorded  during  the  FOTE* 

Summary  Interpretation 

As  an  exploratory  research  effort,  no  definitive  conclusions  were 
warranted  or  intended.  Further,  small  sample  sizes  for  most  critical  task 
performances  preclude  attributing  any  great  degree  of  confidence  in 
statistical  findings.  However,  as  this  research  was  intended  to  suggest 
possible  issues  for  future  MSE  testing  and  provide  insight  into  the  design  of 
personnel  slotting  and  operator  performance  prototypes,  identification  of  the 
more  significant  findings  is  appropriate.  The  following  constitutes  a  summary 
of  the  most  noteworthy  findings. 

o  No  women  were  in  the  31W  MOS,  and  there  were  proportionately  twice  as  many 
women  in  3 IF  than  in  3 ID. 

o  There  was  a  disproportionately  large  number  of  31Ws  in  lower  categories  of 
the  AFQT  (this  may  impact  on  the  quality  of  supervision  and  management). 

o  31Ws  tended  to  have  more  education  than  other  MSE  MOSs. 

o  The  predominant  feeder  MOS  for  31D  and  31F  was  31M;  other  major  feeder 
MOSs  included  31C,  31N,  72E,  and  36M. 

o  The  major  feeder  MOS  for  31W  was  31Y;  31Z  was  also  a  large  contributor. 

o  Very  large  variabilities  in  performance  times  existed  for  the  tasks  of 
troubleshooting,  switch  initialization,  antenna  orientation,  and  NCS/LEN 
set  up  (may  suggest  needs  for  more,  better,  or  different  training  on  these 
tasks) . 

o  For  troubleshooting,  31L  and  31K  MOSs  were  significantly  slower  in  their 
performance  than  most  other  MOSs. 

o  For  switch  initialization,  31C  and  31M  MOSs  clearly  performed  faster  than 
other  MOSs. 

o  For  NCS/LEN  set  ups,  31Ns  did  poorer  than  the  other  major  MSE  feeder  MOSs 
(31M,  72E). 

o  For  LOS  set  ups,  31 Ys  were  clearly  the  poorest  performers. 

o  Significant  correlations  among  personnel  variables  simply  confirm  expected 
relationships  among  these  variables;  however,  there  may  be  a  sex  difference 
on  ASVAB  EL  scores  (women  appear  to  have  made  lower  scores  than  men) . 
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o  While  tenuous  due  to  small  samples,  there  was  an  inverse  relationship 
between  RAU  set  up  and  antenna  orientation  performances  and  between  LOS 
set  up  and  loading  the  KG-94;  overall,  there  was  a  noticeable  absence  of 
relationships  between  tasks  common  to  an  MSE  MOS. 

o  There  were  numerous  significant  relationships  between  certain  personnel 
variables  and  task  performances  (Table  10);  most  of  these  relationships 
were  positive,  the  opposite  of  what  might  be  expected. 

o  Based  on  regressions  of  personnel  variables  on  critical  task  performances, 
troubleshooting,  switch  initialization,  EUB,  NCS/LEN  set  up,  RAU  set  up, 
and  key  loading  appeared  fairly  open  to  prediction;  LOS  set  up,  NCS/LEN  VDU 
operation,  SEN  set  up,  and  antenna  orientation  retained  a  large  proportion 
of  their  variances  as  unpredictable. 

o  There  was  a  significant  improvement  in  communications  procedures  test 
scores  (pre  vs  post  training);  on  both  the  pre  and  post  test,  31Ws 
generally  scored  significantly  better  than  31Ds  or  31Fs,  with  31Ds 
performing  the  poorest. 

o  Provision  of  meaningful  performance  algorithms  for  critical  tasks  was 
restricted  due  to  large  variances  and  relatively  small  samples; 
troubleshooting,  NCS/LEN  set  up,  LOS  set  up,  SEN  set  up,  and  KG-94  key 
loading  appear  as  good  candidates  for  algorithms  (see  Table  13). 

o  For  major  MSE  tasks,  antenna  set  up  and  mast  erection  had  the  highest 

overall  workload  ratings;  the  highest  mental  effort  rating  was  for  31m  mast 
erection  and  the  highest  physical  effort  rating  was  for  antenna  set  up  and 
shelter  set  up  (may  relate  to  manpower  requirements). 

o  More  frequently  reported  operator  errors  included  cabling  problems  and 
mistakes,  poor  generator  use  procedures  and  monitoring,  and  mast  and 
antenna  Installation  problems;  most  other  errors  involved  wrong  or 
accidentally  changed  equipment  settings. 
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